Fast mobile mail with context indicators

ABSTRACT

Facilitating replying to received email messages includes presenting a selectable set of fast reply tokens that correspond to snippets of multimedia content, providing a mechanism to superimpose a selected subset of the fast reply tokens on to different portions of the received email message to compose an annotated email messages, wherein the fast reply tokens are separate from any reply email text, and sending the annotated email message. The fast reply tokens may include action icons, status icons, emoticons, emojis, expressive elements, clip art, highlighters, geometric shapes, arrows, short tunes, audio clips, video clips, abbreviated standard texts, and freehand notes. Facilitating replying to received email messages may also include providing cultural adaptations and localization of fast reply tokens based on a setting provided by a recipient of the annotated email message.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to U.S. Prov. App. No. 61/580,907,filed Dec. 28, 2011, and entitled “FAST MOBILE MAIL,” which isincorporated herein by reference, and claims priority to U.S. Prov. App.No. 61/596,411, filed Feb. 8, 2012, and entitled “FAST MOBILE MAIL WITHCONTEXT INDICATORS,” which is incorporated herein by reference, andclaims priority to U.S. Prov. App. No. 61/624,853, filed Apr. 16, 2012,and entitled “METHOD FOR ROUTING AND ACCESSING CONTENT PROVIDED BY ANAUTHORING APPLICATION,” which is incorporated herein by reference.

TECHNICAL FIELD

This application relates to the fields of mobile Internet communicationsand presentation and editing of multimedia information.

BACKGROUND OF THE INVENTION

Electronic mail remains one of the most popular Internet applications,comparable with online searching. According to industry estimates, therehave been about 3.3 billion email accounts by 2012 used by approximately2 billion users and expected to grow to 4.3 billion accounts by the endof 2016. Over 92% of the adult US population has been using email and66% of those email users do so as part of their typical day. In a broadpicture, email viewing and sending is split between three largeplatforms: desktop, web, and mobile. Mobile email is quickly evolvinginto a dominant communication platform. In mid-2011, desktop platformswere prevailing with the split 53% desktop, 29% web, 18% mobile. ByFebruary 2012, the three platforms have converged and had approximatelyequal market share; starting in April 2012, mobile email has overtakenother platforms, with 38% of emails opened on smartphones, tablets andother mobile devices. According to some estimates, within a coupleyears, mobile email viewing will account for more than 50% of theoverall volume.

Email usage on mobile devices is different from desktop and web mail inmany respects. While viewing and navigating email messages onsmartphones and tablets with a respectable screen size is, for the mostpart, almost as easy and convenient as on the desktop, responding toemails on devices with on-screen keyboards can be a difficult and tiringjob, especially when users are on the go. This puts a strain on some ofthe most active users of mobile email: managers, journalists and othercategories of workers who receive hundreds or even thousands of messagesdaily and need to respond quickly. Some of the challenges of respondingto an email message on a mobile device include: separating fragments ofthe original text from the rest of mail for commenting; typing commentsand edits on soft keyboards; text formatting; adding annotations;identifying annotated materials as part of the original documents, etc.Additionally, existing techniques for transmitting rich content viaemail significantly limits efficient response. For example, when anemail includes attachments opened for editing and markup in non-emailapplications, the attachments may lack routing information which wouldallow automatic sending of edited and annotated fragments of theattachment to all or some recipients directly from the authoringapplication that has opened the attachment. This may cause additionalwork of saving annotated fragments of rich content and re-attaching thefragments to reply emails, which complicates the workflow and presentsadditional challenges to both the author/sender and the recipients.

Accordingly, it is desirable to develop a mechanism for fast andseamless email reply on mobile devices with adequate routing of richcontent and its annotated fragments.

SUMMARY OF THE INVENTION

According to the system described herein, facilitating replying toreceived email messages includes presenting a selectable set of fastreply tokens that correspond to snippets of multimedia content,providing a mechanism to superimpose a selected subset of the fast replytokens on to different portions of the received email message to composean annotated email messages, wherein the fast reply tokens are separatefrom any reply email text, and sending the annotated email message. Thefast reply tokens may include action icons, status icons, emoticons,emojis, expressive elements, clip art, highlighters, geometric shapes,arrows, short tunes, audio clips, video clips, abbreviated standardtexts, and freehand notes. Facilitating replying to received emailmessages may also include providing cultural adaptations andlocalization of fast reply tokens based on a setting provided by arecipient of the annotated email message. The mechanism to superimposethe fast reply tokens may include dragging each of a selected one of thefast reply tokens on to a corresponding portion of the annotated emailmessage. Facilitating replying to received email messages may alsoinclude superimposing a progressing scrolling bar for fast reply tokenson to the one of the received email messages to facilitate efficientnavigation therethrough. Facilitating replying to received emailmessages may also include providing a search bar for searching the fastreply tokens by category. Facilitating replying to received emailmessages may also include automatically restricting a user to sendingonly particular types of input in response to a user being in aparticular setting. Facilitating replying to received email messages mayalso include automatically restricting a user to receiving onlyparticular types of input in response to a user being in a particularsetting. A user may be restricted to sending and receiving audio-onlyinput in response to the user being in a moving automobile. Facilitatingreplying to received email messages may also include automaticallyconverting a message from a first type of input to one of the particulartypes of input, wherein the first type of input is not one of theparticular types of input. Facilitating replying to received emailmessages may also include determining an overall tone of the annotatedemail message based on the fast reply tokens superimposed on to theannotated email message and providing to a recipient an indication ofthe overall tone of the annotated email message. Facilitating replyingto received email messages may also include including contextinformation with at least some of the fast reply tokens superimposed onto the annotated email message, wherein the context information includesportions of source documents associated with at least some of the fastreply tokens. Each of the portions of source documents may includelocation information indicating a location within the document for thecorresponding portion. The location information may indicate a page anda position on the page for the corresponding portion. The position onthe page may be descriptive or may be indicated using a graphicalmatrix.

According further to the system described herein, computer software,provided in a non-transitory computer readable medium, facilitatesreplying to received email messages. The software includes executablecode that presents a selectable set of fast reply tokens that correspondto snippets of multimedia content, executable code that superimposes aselected subset of the fast reply tokens on to different portions of thereceived email message to compose an annotated email messages, where thefast reply tokens are separate from any reply email text, and executablecode that sends the annotated email message. The fast reply tokens mayinclude action icons, status icons, emoticons, emojis, expressiveelements, clip art, highlighters, geometric shapes, arrows, short tunes,audio clips, video clips, abbreviated standard texts, and freehandnotes. The software may further include executable code that providescultural adaptations and localization of fast reply tokens based on asetting provided by a recipient of the annotated email message. Themechanism to superimpose the fast reply tokens may include dragging eachof a selected one of the fast reply tokens on to a corresponding portionof the annotated email message. The software may further includeexecutable code that superimposes a progressing scrolling bar for fastreply tokens on to the one of the received email messages to facilitateefficient navigation therethrough. The software may further includeexecutable code that provides a search bar for searching the fast replytokens by category. The software may further include executable codethat automatically restricts a user to sending only particular types ofinput in response to a user being in a particular setting. The softwaremay further include executable code that automatically restricts a userto receiving only particular types of input in response to a user beingin a particular setting. A user may be restricted to sending andreceiving audio-only input in response to the user being in a movingautomobile. The software may further include executable code thatautomatically converts a message from a first type of input to one ofthe particular types of input, wherein the first type of input is notone of the particular types of input. The software may further includeexecutable code that determines an overall tone of the annotated emailmessage based on the fast reply tokens superimposed on to the annotatedemail message and executable code that provides to a recipient anindication of the overall tone of the annotated email message. Thesoftware may further include executable code that includes contextinformation with at least some of the fast reply tokens superimposed onto the annotated email message, wherein the context information includesportions of source documents associated with at least some of the fastreply tokens. Each of the portions of source documents may includelocation information indicating a location within the document for thecorresponding portion. The location information may indicate a page anda position on the page for the corresponding portion. The position onthe page may be descriptive or may be indicated using a graphicalmatrix.

According further to the system described herein, sharing modificationsof a document among a plurality of authors includes publishing thedocument using a service that is accessible by at least some of theauthors and propagating modifications by a first one of the authors toother ones of the authors by sending change information to the other oneof the authors, wherein the change information includes a pointer to thedocument, an indication of the modifications, and an identification ofthe first one of the authors and wherein the change information does notinclude the document. The service may be a cloud service. Changeinformation may be sent using email. The identification of the first oneof the authors may include a routing ID of the first one of the authors,where the routing ID facilitates locating the first one of the authorson the Internet. The routing ID may be an email address of the first oneof the authors. Modifications to the document may be performed using anauthoring application. Less than all of the plurality of authors mayhave the authoring application. The document may be viewable by all ofthe authors using an email client. Modifications to the document may beperformed using an authoring application. The change information mayidentify the authoring application. Sharing modifications of a documentamong a plurality of authors may also include a second one of theauthors, different from the first one of the authors, performingadditional modifications to the document and propagating the additionalmodifications by the second one of the authors to other ones of theauthors by sending supplemented change information to the other one ofthe authors, where the supplemented change information appends to thechange information an indication of the modifications and anidentification of the second one of the authors. The change informationmay include an indication of particular modifications made by each ofthe authors. Sharing modifications of a document among a plurality ofauthors may include supplementing the change information withidentification information for additional recipients, where at least oneof the additional recipients does not perform any modifications to thedocument.

According further to the system described herein, computer software,provided in a non-transitory computer readable medium, sharesmodifications of a document among a plurality of authors. The softwareincludes executable code that publishes the document using a servicethat is accessible by at least some of the authors and executable codethat propagates modifications by a first one of the authors to otherones of the authors by sending change information to the other one ofthe authors, where the change information includes a pointer to thedocument, an indication of the modifications, and an identification ofthe first one of the authors and where the change information does notinclude the document. The service may be a cloud service. Changeinformation may be sent using email. The identification of the first oneof the authors may include a routing ID of the first one of the authors,where the routing ID facilitates locating the first one of the authorson the Internet. The routing ID may be an email address of the first oneof the authors. Modifications to the document may be performed using anauthoring application. Less than all of the plurality of authors mayhave the authoring application. The document may be viewable by all ofthe authors using an email client. Modifications to the document may beperformed using an authoring application. The change information mayidentify the authoring application. The software may also includeexecutable code that propagates additional modifications by a second oneof the authors to other ones of the authors by sending supplementedchange information to the other one of the authors, where thesupplemented change information appends to the change information anindication of the modifications and an identification of the second oneof the authors. The change information may include an indication ofparticular modifications made by each of the authors. The computersoftware may also include executable code that supplements the changeinformation with identification information for additional recipients,where at least one of the additional recipients does not perform anymodifications to the document.

Replying to email on a mobile device includes a responder using fastreply tokens, which represent snippets of multimedia content overlaidupon the original text and in line with the previously added fast replytokens. Fast reply tokens may be positional or generic and may include:commonly accepted action and status icons and symbols; expressiveelements and representations of mood, such as emoticons and/or emojis,clipart from pre-compiled or other available libraries, highlighters,geometric shapes for annotations, arrows, connectors, pointers and othermarkup and annotation elements, short tunes, audio and video clips thatmay be pre-recorded and retrieved from a library or recorded by theresponder in the process of replying, abbreviated standard texts,freehand writing and drawing, and/or small fragments of typed text.

Fast reply tokens may be applied to selected elements in the originalemail such as words, phrases, paragraphs of text, images, etc.Alternatively, fast reply tokens may pertain to the whole message. Fastreply tokens may be made available either inline, with the text flow ofthe original email message, for example, within callouts associated withthe selected elements or the tokens may be inserted into anautomatically or manually created blank space in the body of an email.Tokens may be added to an email response using tap-and-select userinterface (UI), multi-touch gestures or other UI methods. In oneembodiment, a dedicated multi-touch gesture (such as swiping) may invokethe token toolbox for user choice.

The appearance and features of a fast mobile mail message received by auser may differ depending on the device to which the message arrives.Message adaptation and conversion occurs on the server and is part ofthe fast mobile mail cloud service which stores user profiles and inparticular profiles and desired presentation modes for each device ownedby the user. Message adaptation may include conversion of fast replytokens between diverse media types. In an embodiment, if the user drivesa car and receives a fast mobile mail on a smartphone by downloading itfrom the server, the system may automatically detect the smartphone as areceiving device and the user driving status, whether the latter hasbeen previously set by the user or automatically detected by the systemvia interaction of smartphone with the built-in car electronics. Undersuch conditions, the system may transmit the fast mobile mail messagewith all visual elements converted into their audio representations, sothat the user does not have to look at a visual display while drivingand will still get the most of message content. Alternatively, a userhaving hearing issues or receiving a fast mobile mail message in a noisyenvironment may pre-configure fast mobile mail so that all audioelements will be converted by the fast mobile mail service into theirvisual equivalents. In another embodiment, a user who receives a fastmobile mail message on a device without video capabilities maypre-configure the personal fast mobile mail account so that video fastreply tokens are disabled or replaced them with one or a series ofcharacteristic static screens that can be viewed by the user.

The fast mobile mail may offer cultural adaptation, such as automatictranslation of the text to deliver text snippets to each recipient inthe preferred language, filtering, on user demand, suspicious textual,graphical and audio content arriving to the user, depending on culturalpreferences, transcribing visual fast reply tokens in accordance withlanguage and cultural settings, for example, converting emoticons intothe corresponding emoji during an English-to-Japanese translation of afast mobile mail message, and vice versa, and/or delivering contact infoto each recipient in the desired language and with the appropriaterecipient country specific formatting of contact fields.

Fast mobile mail may include device software for a variety of mobileplatforms and cloud service. In an embodiment, fast mobile mail may belimited to the device software, which may be designed as aplug-in/add-on software application to existing mobile mail clientsoftware applications. The cloud service associated with the fast mobilemail may be a part of broader online note-taking or personal contentmanagement services such as Evernote developed by Evernote Corporationof Redwood City, Calif., USA. The cloud service may securely storeinformation about registered users and profiles and preferences of usersand may also store fast mobile mail messages for push or pull access byusers. In one embodiment, the cloud service includes (or communicateswith) third party mail and/or instant messaging servers (such as SMTP orXMPP servers), document storage systems, etc. In another embodiment, thedevice software interacts with third party mail and instant messagingsoftware on its own. Hybrid interaction with the third party mail andinstant messaging software is also possible.

The device software may work offline, disconnected from the associatedcloud service; online—permanently connected with the service, andpartially online when connection to the cloud service is availableperiodically. Functions of fast mobile mail may be distributed betweendevice software and cloud service, depending on the connection scenarioand parameters.

Upon completion by a responder of a fast mobile mail message, devicesoftware and/or cloud service may analyze fast reply tokens and attemptto build an abstract of the message, such as defining an overall mood ofthe communication by counting fast reply tokens with emotionallymeaningful content, including such tokens as emoticons and standardtunes, emphatic words in text snippets, etc. If a prevailing mood (tone)of a fast mobile mail message can be recovered with a sufficientcertainty then the mobile software and/or cloud service may add a visualrepresentation, an audio representation or both of the abstract in theform of a brief tune, image, animation or other display consistent withthe mood of the message and replayed to each recipient (who hasappropriately configured fast mobile mail to receive such introductions)prior to displaying the message. Similar to fast reply tokens, abstractsmay be culturally adapted, as explained elsewhere herein.

When a fast mobile mail message, authored by a user with device softwareinstalled, arrives to a recipient who also has device softwareinstalled, the recipient can view the full message and replay all fastreply tokens, subject only to the capabilities of the receiving device.However, when a recipient does not have the device software installedand receives a fast mobile mail message via regular email, the messagemay be displayed in an adapted format, such as a standard HTML or animage rendered with certain layout and disposition of fast reply tokens.In this case, full multimedia copy of the fast mobile mail message maybe stored at the cloud service and the adapted message delivered to therecipient may include a link to the full copy which the recipient mayuse after installing the device software.

The device software may provide a set of multimedia methods forauthoring mail replies, editing documents, communicating with the cloudservice and, when necessary, with other third party software andservices, including traditional email software. Functions of the devicesoftware may include one or more of the following:

Rendering, displaying and navigating incoming mail message on the devicescreen.

Attaching fast reply tokens to words or other semantic elements inline,within the text flow, or adding free space for replies.

Quick navigation of fast mobile mail message, such as scanning fastreply tokens by entry time, spatial arrangement, category of content,etc.

Sending fast mobile mail reply to the cloud service (when connected) forsubsequent distribution to recipients and storage in the cloud service,as explained elsewhere herein; also, synchronizing local storage withthe device.

Receiving fast mobile mail reply from another user by communicating withthe cloud service or other content distribution source.

Receiving, rendering and playing an abstract or introductory tune to areply received from another user, according to receiving user's profileand preferences.

Search in fast mobile mail message contents by differentcharacteristics.

The cloud service may provide a set of mechanisms for accessing,directing and controlling the content of original documents and replies,controlling device software and delivering fast mobile mail messages.Functions of the cloud service may include one or more of the following:

Storing user accounts, profiles, settings, and sending/receivingpreferences adapted to user devices, languages, location, culturalbackground, disposition, and other situational parameters.

Keeping and employing sets of dictionaries and lists used forsituational and cultural adaptation and conversion (transcoding) of fastreply tokens.

Receiving original email message or document from a mail server or othersource and sending the message to the device software for display andreply on a user device. It should be noted that, alternatively, theemail message/document can be delivered to the user device directly by athird party mail or other content distribution service and opened in thedevice software serving as a plugin to third party software, withoutparticipation of the cloud service.

Synchronizing user preferences and settings with each user device, whichmay be subsequently employed by the device software running on thatdevice.

Updating device software when new versions are available.

Receiving and storing copies of the original fast mobile mail messagesfrom the device software and of the adapted copies built in accordancewith recipient preferences and their device capabilities; generatinglinks to access stored copies.

Customizing fast mobile mail messages and copies of edited documentsaccording to recipient preferences and their device capabilities.

Analyzing fast mobile mail messages and creating abstracts in the formof introductory or supplementary tunes or visuals.

Sending replies and edited documents to recipients in custom formatsusing embedded mail server or communicating with third party mailingsystems and other content distribution means.

Displaying adapted version of replies and edited documents to users uponrequest in a web browser. This option applies to recipients whoseprincipal mail client cannot render fast mobile mail messages and whodon't have device software installed on their devices.

It should be noted that carrying out the above techniques may requiredifferent degrees of connectivity between the device software, the cloudservice and third party email and other content distribution services.For example, sending a fast mobile mail message from a user device mayrequire instant connection and transmission of content, whilesynchronization or changing of a user profile may require only periodicconnection between with the cloud service. In the event when a userdevice remains offline, i.e. disconnected from the cloud service forprolonged periods of time, the device software may partially duplicatesome of the functions of the cloud service and synchronize the changeswith the cloud service once the connection becomes available.

Fast mobile mail messages may include annotated snippets of documentsattached to the original email messages or otherwise delivered to aresponder. In the event that document formatting has a fixed pagination(for example, a PDF document) it may be desirable to provide torecipients a detailed reference to snippet location in the document tofacilitate understanding the context of the annotations. Since a snippetcan be significantly smaller than the page it is extracted from, areferencing mechanism may be expanded to the sub-page level. A proposedtechnique for referencing portions of documents with fixed paginationincludes context indicators, which are a visual reference attached to anannotated snippet in the form of a footer, header, callout, overlay orother visible element, which includes the document name (which may be amore generic URI—Unified Resource Identifier), the page number withinthe document, and a graphical matrix showing parts of the page where thesnipped location(s) is (are) highlighted. Thus, a 3×3 matrix mayindicate nine parts of the page from the bottom left part to the topright part, and a highlighted matrix element within the contextindicator may indicate, for example, snippet location in the rightbottom part of the page or two highlighted elements may place a snippetacross the middle left and the top left parts. Alternatively, details ofsnippet location on the page may be represented textually; thus, in theprevious example, instead of displaying a 3×3 matrix and highlightingdifferent elements of the matrix, the context indicator may augment thedocument and page reference with the words “top right of the page” or“middle left and top left of the page”.

As explained elsewhere herein, annotated documents and portions thereof,obtained through email attachments, downloads and other contentdistribution means, including fast mobile mail messages and portionsthereof, may be saved in the cloud service for subsequent access byother users or by the original authors. Existing authoring applicationsmay not store information allowing correct routing of such modifieddocuments. In order to support a natural routing process between allsenders and recipients of the multimedia content related to fast mobilemail, the device software or the cloud service may add sender's routingidentifier (routing ID), such as email address, to the Unified ResourceIdentifier (URI) of the saved location, i.e. to its access link.Furthermore, every transmission of the access link to recipients mayinclude the routing ID of the sender and may be further augmented withrouting IDs of all users who have subsequently edited the content andsaved it to the cloud service. The URI may also be augmented withrouting IDs of all recipients or only the recipients that have viewedthe content. In this way, content exchange associated with fast mobilemail but not necessarily transmitted via email may automatically buildseveral user groups associated with the content. These groups mayinclude, for example, authors (users who have actually modified thecontent), recipients (everyone who has received the links to content),and viewers (users who have downloaded the content from received linksto content locations). In particular, it becomes possible to send all ofthe authoring content that has been created in an authoring application.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the system described herein will now be explained in moredetail in accordance with the figures of the drawings, which are brieflydescribed as follows.

FIGS. 1A-1C are schematic illustrations of fast reply tokens in fastmobile mail messages according to an embodiment of the system describedherein.

FIGS. 2A-2C illustrate navigation (scrolling) and search results withina fast mobile mail message according to embodiments of the systemdescribed herein.

FIGS. 3A-3B illustrate two system architecture charts and interactionsbetween device software, cloud service and conventional mail systemaccording to embodiments of the system described herein.

FIGS. 4A-4B are schematic illustrations of various adaptations of fastmobile mail messages to recipient devices and user defined optionsaccording to an embodiment of the system described herein.

FIG. 5 illustrates creation and replay of an abstract summarizing a fastmobile mail message according to an embodiment of the system describedherein.

FIG. 6 is a schematic illustration of context indicators for referencingdocument snippets according to an embodiment of the system describedherein.

FIG. 7 is a content access and routing chart for documents and portionsthereof saved to the cloud service from authoring applications such asfast mobile mail and accessible via content links according to anembodiment of the system described herein.

FIGS. 8A-8D are system flow diagrams that describe processing associatedwith different embodiments of the system described herein.

DETAILED DESCRIPTION OF VARIOUS EMBODIMENTS

The system described herein provides a mechanism for communicationsbetween mobile users whereby replying to an email on a mobile deviceincludes a responder applying fast reply tokens, which representsnippets of multimedia content overlaid upon the original text and inline with the previously added fast reply tokens. The system augmentsand in many instances replaces a traditional mail where responding to amessage involves heavy typing, conflicting with the contemporary mobileusage metaphor, especially on keyboardless smartphones and tablets.

FIGS. 1A-1C are schematic illustrations 100 of various types of fastreply tokens applied by a responder within fast mobile mail messagesaccording to an embodiment of the system described herein. A deletiontoken 110 is a recommendation by the responder to delete a word. Invarious embodiments, it may be either a markup sign or an active buttonattached to the underlying content that deletes the word upon beingpressed by a recipient of the fast mobile mail message. A highlightedtext token 115 is annotated by an audio clip token 120 represented as acallout with symbolic icon; the clip may be replayed upon clicking onthe note icon portion of the token 120. A callout-shaped smiling faceemoticon token is illustrated as being attached to the word “viverra” inthe original text of the message. A brief text note token 130 iscollapsed on FIGS. 1A-1B and is a comment to the word “libero” in theoriginal message, as shown by a connector 135. All the fast reply tokens110, 115, 120, 125, 130, 135 are also shown on FIG. 1B, which adds otherannotation elements—a pointing arrow token 140 and an overlaid textcomment token 145 applicable to a word pointed to by the arrow. Itshould be noted that all fast reply tokens 110, 115, 120, 125, 130, 135,140, 145 are positional inline elements that are tied to specificelements of the original message, such as words and highlighted textfragments, and add comments to the specific elements or express aresponder's mood with respect to the specific elements. FIG. 3C showsthree more types of positional inline markup elements—a fragment offreehand writing (underline, in this case) token 150, an encirclingshape token 155 (which may be chosen from a broad shape palette), and aclipart icon token 160 that is attached to a word in the text by aconnector/pointing arrow token 165. In an embodiment of the systemdescribed herein, the fast reply tokens 110, 115, 120, 125, 130, 135,140, 145, 150, 155, 160, 165 may all be displayed on top of the originalmessage content and do not necessarily require reformatting and addingblank space to the original message.

Another option is illustrated by two unattached fast reply tokens—avideo clip icon token 170 and a thumbs up (approval) emoticon token 175.The content of the video clip icon token 170 and an emotion or moodassociated with the emoticon token 175 may apply to the above or thefollowing paragraph in the original message or both depending on thecontext and responder intent. In different embodiments, techniques toadd free space and to insert new elements into the original message mayvary. In one embodiment, fast reply tokens may be chosen from a specialtoolbox of unattached tokens, and dropping them at a certain position inthe message may automatically create enough free space (for example,lines of text below the dropping position are moved down) to insert thetoken.

Because fast reply tokens are applied to different portions of themessage and have different display formats, efficient navigation andsearch in fast mobile mail is useful. FIGS. 2A-2C are a schematicillustration of navigation (progressive scrolling) of a fast mobile mailmessage and of search capabilities of fast mobile mail software. Aprogressive scrolling button 210 on FIG. 2A is part of a user interfacefor fast mobile mail; tapping the button 210 moves the message down (orup) and selects a next fast reply token. As illustrated on FIG. 2A, acurrently selected fast reply token 220 is a callout with an audio clip(selection is shown here, for the explanatory purpose, by invertingcolors of the token). The fast reply token 220 is placed by the devicesoftware, for better visibility, in a top portion of the mobile devicescreen, near the middle thereof. A next fast reply token 230 on the leftscreen of FIG. 2A is a collapsed text note with a connector. Pressingthe progressive scrolling button 210 in the down direction scrolls themessage down, selects the fast reply token 230 (the fast reply token 230is shown in inverted colors in the right screen) and positions theportion of the message corresponding to the fast reply token 230 forconvenient viewing on the screen, as shown in the right part of FIG. 2A.Subsequently, pressing the down portion of the progressive scrollingbutton 210 would select and re-position a film strip fast reply token250, while pressing the up portion of the progressive scrolling button210 would reverse the navigation and return the screen to an initialposition showing the selected fast reply token 220. Generally, thescrolling button allows scrolling between instances of fast reply tokesin a message.

Searching in fast mobile mail messages is illustrated on FIGS. 2B-2C. Asearch field 260 directs searching by categories, keywords and/or otherattributes and content elements of fast reply tokens. For example, thesearch field 260 of FIG. 2B searches by fast reply token categories“emoticons” or “clipart” and is applied to the message illustrated onFIGS. 1A-1C, to result in three tokens 270 on the visible screen: aninline smiling face emoticon token, an unattached thumbs up emoticontoken, and a clipart token. Of course, it is possible that the rest ofthe fast mobile mail message contains other fast reply token satisfyingthe search term and accessible via progressive scrolling button 275.FIG. 2C illustrates search results in fast mobile mail message byanother search term 280, combining categories audio and video of fastreply tokens to retrieve two fast reply tokens 290 with an audio andvideo clip. Other search results (if any) may be reached using buttons295.

FIGS. 3A-3B are system architecture charts, showing interactions betweendevice software, cloud service and a conventional mail system accordingto embodiments of the system described herein. FIG. 3A illustrates anembodiment of the system where traditional mail software is functioningseparately from the fast mobile mail service and is interacting with thecloud service, which is part of fast mobile mail. A fast mobile mailmessage 305 is created by device software 310 by adding fast replytokens to an original regular mail message 340 delivered to the regularmail client running on the mobile device by a mail server 330 (which mayalso be an XMPP instant messaging server or other message deliverysolution). The device software 310 utilizes user settings stored in acloud service 320 and saves completed fast mobile mail messages, alongwith a recipient list designated by the sender, to the network storagepart of the cloud service. Subsequently, the cloud service 320 analyzesreceived fast mobile mail messages and their recipient lists, preparesnecessary delivery options, adaptations, links to stored content andother necessary data, forms regular emails with links to stored content(where necessary), and delivers, to the mail server 330, regular emailmessages addressed to appropriate recipients. The mail server 330 thenroutes the email messages to recipients, possibly using the samechannel. In the architecture of FIG. 3A, an authoring part of fastmobile mail 310 is separated from the mail delivery channel and the mailserver 330. At the same time, the device software 310 is capable ofdownloading fast mobile mail content directly from the network storageof the cloud service 320; such downloading may occur every time arecipient (or a sender) of a fast mobile mail message needs to accessall details of fast reply tokens, which may not be available usingregular email software. As explained elsewhere herein, fast mobile mailmessages delivered via regular email may be rendered, to a certainextent, by regular mail client software; however, full rendering,progressive scrolling and search capabilities, editing and adding newfast reply tokens may be available only in the device software, whichdownloads full message content from the cloud service or extracts fullmessage content from attached or embedded additional data in the regularemail message.

FIG. 3B illustrates another embodiment of the system architecture wheremail server and mail client software are tightly integrated with thecloud service and device software. In the embodiment of FIG. 3B, a mailserver 380 functions as part of a cloud service 370 and delivers emailsor documents 390 directly to device software 360, which combinesfunctions of mail client software and fast mobile mail authoringsoftware. In this architecture, all fast mobile mail messages 350 may berendered and authored using a single software module running on thedevice (i.e., the device software 360).

FIGS. 4A-4B illustrate two different adaptation scenarios for fastmobile mail messages, as explained elsewhere herein. FIG. 4A illustratescultural adaptation and localization of a fast mobile mail message,while FIG. 4B shows adaptation of a message to special receivingconditions. After the fast mobile mail message has been created with aplurality of original fast reply tokens 402, 405, 410, 412, the fastmobile mail message is transmitted as a data package 415 that issupplied with a list of recipients to a cloud service 420 (in thisexample, the cloud service 420 combines fast mobile mail service andmail delivery functions, as explained elsewhere herein). Subsequently,the cloud service 420 analyzes the content and the list of recipients,along with fast mobile mail settings 422 for each recipient. In thisexample, the recipient settings 422 require adaptation of the fastmobile mail message to Japanese language and to a correspondingculturally acceptable appearance. The cloud service 420 performs suchadaptation and delivers the adapted fast mobile mail message 425 to therecipient.

In of FIG. 4A, the following conversions have been made in theadaptation process:

(1A) the original text of the email message has been translated toJapanese;

(2A) a deletion fast reply token 402 and a sound clip fast reply token403 have been reproduced in their original form as fast reply tokens432, 433, except the markup to which the sound clip fast reply token 433has been attached has migrated to the appropriate portion of theJapanese text;

(3A) a comment fast reply token 405 has been translated to a Japaneseequivalent fast reply token 435 and reproduced with a similar pointingarrow in the adapted fast mobile mail message;

(4A) a text of note fast reply token 412 (which is shown in thecollapsed form on the screen and the text is reproduced on anexplication) has been translated to Japanese and an equivalentdestination fast reply token 442 has been created;

(5A) an emoticon fast reply token 410 has been converted into atraditional Japanese emoji fast reply token 440 with a similar emotionalmeaning; the emoji was attached to the corresponding word in thetranslated text.

FIG. 4B illustrates an adaptation where an original, predominantlyvisual fast mobile mail message with certain audio elements is convertedinto an audio-only fast mobile mail message for a recipient whosesettings require such adaptation at the time of message delivery (forexample, the recipient is driving a car when receiving the message). Theillustration shows adaptation of only part of the fast reply tokens; therest of the adaptation is analogous. After the fast mobile mail messagewith fast reply tokens has been created, the fast mobile mail message istransmitted as a data package 465 to a cloud service 470, which analyzesthe message content and recipients, as explained elsewhere herein. Inthis example, one of the recipients may have enabled a frequent driveroption in settings 472, which provides for checking user status beforedelivering fast mobile mail messages. Accordingly, prior to deliveringthe message, the cloud service 470, which is connected to the mobiledevice as illustrated by a dashed arrow 475, discovers that therecipient status indicates that the recipient is driving and thatimmediate delivery is required, which implies adaptation of the fastmobile mail message to an audio format. The cloud service performs suchadaptation and delivers an audio message 480 to a mobile device of therecipient. The audio message is subsequently replayed to the recipient.Specific adaptations of the original fast reply tokens are as follows:

(1B) text-to-speech conversion of the original message text (before fastreply tokens have been added) has been performed and the speech portionhas been read to the recipient with modifications (described below);

(2B) each words or fragment of text to which a fast reply tokens isattached may be emphasized by audio volume, intonation, tempo or otheraudio means, as illustrated by bold text 482, 490, 492 and 495;

(3B) after each anchor word, a brief separator tune 487 may be replayed,in order to separate the body text (original message) from audiomessages associated with fast reply tokens;

(4B) each fast reply token may be converted to a specific voice messageand reproduced after a separator tune, concluded with another separatortune, identical to or different from the starting separator tune. Forexample, the deletion icon fast reply token 450 may be converted into avoice message 488 that says: “delete ‘eget’”, an audio tune 452 may bereproduced after an anchor fragment 490 without changes and may beseparated from the speech by the separator tunes 487, a comment 455 maybe read as a voice message 493, separated from an anchor word 492 andfrom the rest of the audio content of the fast mobile mail message byseparator tunes, an emoticon 460 may be replaced with an audioequivalent, a smiling tune 496, which sounds after an anchor word 495and the starting separator tune.

FIG. 5 is an illustration 500 of creation and transmission of anabstract for a fast mobile mail message. Device software 510 running ona mobile device of a sender may analyze fast reply tokens and determinewhether an abstract for the fast mobile mail message can be built. Inthis example, the device software 510 may analyze a first set of fastreply tokens 520 that tend to convey emotions and a second set of fastreply tokens 530 that tend to be emotionally neutral and decide that theoverall tone of the message is, for example, positive and smiling (orsimilar). After the determination, the device software 510 may selectsan abstract in the form of a happy tune (or similar) 540 or in othervisual or audible content, attach the abstract to the fast mobile mailmessage, and transmit the fast mobile mail message with the abstract viaa cloud service 550 to device software 560 running on a mobile device ofa recipient. The recipient may decide whether to replay the abstractprior to displaying the fast mobile mail message, which may cause thescreen 560 the received fast mobile mail message to be dimmed. Therecipient may elect to read immediately, and then possibly defer ordelete the received message based on the abstract.

FIG. 6 is an illustration 600 of context indicators for documentsnippets delivered within a fast mobile mail message. A fast mobile mailmessage 610 contains two snippets, extracted from one common PDFdocument. A first snippet is annotated with a fast reply token 620containing a question, whereas another snippet includes a meeting anapproval sign, an emoticon fast reply token 630. In order to adequatelylocate the snippets in the source document, context indicators 640 maybe appended to each snippet showing not only the source document and thepage from which the snippet was clipped but also the location of thesnippet on the page, illustrated by locators on corresponding graphicalmatrixes 640, 641. Thus, the snippet in the first example was extractedfrom the bottom right corner of page 3 of the document, as indicated bythe matrix 640, whereas in the second example, the snippet was clippedfrom the middle top portion of page 16 of the same document, asindicated by the matrix 641.

FIG. 7 is a content access and routing chart 700 for documents and othercontent created and edited by authoring applications and stored in thecloud. The documents and other content may be available via downloadlinks to multiple recipients, authors and viewers and may besimultaneously distributed via email and other conventional contentdelivery services. A proposed method of content publishing and accesstakes advantage of enhanced routing technique, as explained elsewhereherein. An authoring application (AuthApp) 710 creates original content715 and publishes the original content on a cloud-based contentmanagement service 720. The AuthApp 710 may also create a customUniversal Resource Identifier (URI) 725 for accessing the publishedcontent. The custom URI 725 consists of an identifier or name of theauthoring application 710, location of the published content (such as aweb URL), and a routing ID of the publisher, or sender, of the content,which can be an email or other routing address of the sender thatlocates the sender on the Internet.

The custom URI 725 may be distributed via traditional content deliveryservices, such as email. In this example, content created by theauthoring application 710 may be fully rendered by regular mailsoftware. After the URI 725 is delivered to another authoringapplication (AuthApp) 730, the other authoring application 730 retrievesand edits the original content and creates modified content 735, such asa fast mobile mail message. In some cases, regular mail software may notbe capable of full rendering of the modified content 735. After theother authoring application 730 saves the modified content to the cloudand creates a second custom URI, which may include routing IDs of boththe original sender and the recipient (who has become the secondauthor), the AuthApp 730 analyzes the modified content, discovers thatthe modified content cannot be rendered by a regular email and, as aresult, creates a separate content instance for rendering in an email,which may be as close as possible to fully displayed content in theauthoring application itself.

The separate content instance may be in the form of HTML mail, embeddedpre-rendered image, etc. The AuthApp 730 may then add the separatecontent instance of the modified content to the body of an emailmessage, along with the second custom URI for the modified content. TheAuthApp 730 may then transmit the message to mail client software 740,which renders the (simplified) separate content instance to providerendered content 745 and transmits the rendered content 745, via a mailserver 750 and other parts of a mailing system, to subsequent recipients(the original sender and/or other recipients). The recipients may viewthe rendered content using mail software. The recipients may alsoreceive custom URIs, which, if the recipients also have authoringsoftware installed and capable of processing full content, therecipients may use to download, fully render and further modify thecontent. In an embodiment, authoring applications that further modifyand distribute content may add not only a routing ID corresponding tothe authoring application, but also routing IDs of subsequentrecipients. This technique builds up groups of authors, recipients andviewers simultaneously with content distribution, as explained elsewhereherein. Note that, in some cases, it may be possible to send the changeswithout sending the content. Note also that it is possible to make adocument accessible over the Internet using a mechanism other than cloudstorage.

Referring to FIG. 8A, a flow diagram 800 illustrates processingperformed in connection with user activities with fast mobile mail.Processing starts at a step 801 where a user signs up for the fastmobile mail service. After the step 801, processing proceeds to a step802, where the user installs the fast mobile mail software, that is, thedevice software, as explained elsewhere herein. After the step 802,processing proceeds to a step 803, where the user fills in settings andconfiguration for fast mobile mail software and service, as explainedelsewhere herein. Settings may be stored under an account of the user ina cloud service and may be fully or partially duplicated on mobiledevice(s) where the user runs fast mobile mail.

After the step 803, processing proceeds to a step 804, where the userreceives a fast mobile mail message or downloads corresponding contentfrom network storage using links provided via email or other means, suchas custom URIs, as explained elsewhere herein (see in particular FIG. 7,described above). It should be noted that the user may also receive anordinary mail message from mail accounts that do not have fast mobilemail installed or do not use fast mobile mail to enhance communication.Depending on a particular embodiment of the system architecture,software that receives the message on the mobile device of the user maybe regular mail client software that communicates with the devicesoftware or the device software that combines the fast mobile mailauthoring function with the mail client capabilities (see in particularFIGS. 3A-3B, described above). After the step 804, processing proceedsto a step 805, where the user views, navigates and searches the contentof the received fast mobile mail message, as explained elsewhere herein(see, for example, FIGS. 2A-2C, 5, and 6, described above). After thestep 805, processing proceeds to an optional step 806, where the useredits the received mail message and replies to/with a first (in case theoriginal message was an ordinary mail message) or another fast mobilemail message (see, for example, FIGS. 1A-1C, described above). In caseswhere the user responded to the received message, after the step 806,processing proceeds to a step 807, where the user sends the created fastmobile mail message and/or publishes a copy of the message or copies ofportions of content of the message to the cloud service, as explainedelsewhere herein (see, for example, FIGS. 3A-3B, 4A-4B, 5, and 7,described above). It should be noted that the system may automaticallypublish any needed copies to the cloud service based on user settings oron analysis of the message content. Following the step 807, processingis complete.

Referring to FIG. 8B, a flow diagram 810 illustrates in more detailprocessing provided at the step 804 where the user receives or downloadsfast mobile mail message and/or its content. The user receiving fastmobile mail messages may be fulfilled via different services, softwareand user activities and may also depend on the system architecture, asexplained elsewhere herein. Processing starts at a test step 811 whereit is determined whether the user intends to manually download thecontent of a fast mobile mail message using, for example, a receivedcustom URI like the custom URI 725 described elsewhere herein. If so,then control transfers from the step 811 to a step 813 where the userdownloads the needed content. Following the step 813, processing iscomplete.

If it is determined at the step 811 that the user does not intend tomanually download the content of a fast mobile mail message, thencontrol transfers from the step 811 to a step 812 where the cloudservice checks recipient's mobile settings to determine delivery formatand options for the fast mobile mail message and/or the downloadablecontent. All subsequent steps on the flow diagram 810 pertain to thecloud service. After the step 812, control transfers to a test step 814,where it is determined whether content adaptation is needed for therecipient. If so, then control transfers to a step 815 where the cloudservice creates an adapted copy of the fast mobile mail message. Asdiscussed elsewhere herein, message adaptation may include one or moreof the following: language translation of textual and/or audioinformation; cultural adaptation, such as replacing emoticons andclipart with their culturally acceptable instances based on userpreferences; converting fast reply tokens or the whole fast mobile mailmessage into different formats, such as audio-only or visual-onlyformats, etc. (see, for example, FIGS. 4A-4B, described above).

If it is determined at the test step 814 that there is no need foradaptation, then control transfers to a test step 816. Note that thetest step 816 is also reached from the step 815, described above. At thetest step 816; the cloud service determines, based on the recipient usersettings, whether the device software is installed on the target mobiledevice for message delivery. If so, then control transfers to a step 818where the cloud service sends the full content of the fast mobile mailmessage to the recipient's device since the message can be rendered inits entirety by the installed device software. Following the step 818,processing is complete.

If it is determined at the test step 816 that the recipient does nothave appropriate device software installed, then the control transfersfrom the step 816 to a step 817. At the step 817, the cloud serviceprepares a renderable instance of the fast mobile mail message fordelivery by a regular electronic mail. Producing a renderable instancemay include providing an HTML version or pre-rendered image(s) of thefast mobile mail message (potentially adapted at the step 815). At thestep 817, the cloud service may publish full content of the fast mobilemail message to its cloud storage, creates a link to that content, sucha custom URI, and augments the message with a link to the custom URI,which enables the recipient to receive the full fast mobile mail messageon a different mobile device or on the same device after installing thedevice software at a later time. After the step 817, control transfersto a step 819, where the cloud service sends, via regular email, asimplified version of the fast mobile message and the custom URI link tothe published copy of the full content. Following the step 819,processing is complete.

Referring to FIG. 8C, a flow diagram 820 illustrates in more detailprocessing provided at the step 805 where the user replays, navigatesand searches the content of the received fast mobile mail message.Processing starts at a test step 821 where it is determined whetherdevice software is installed on the user's mobile device to enableplaying fast mobile mail messages. If so, then control transfers fromthe test step 821 to a step 824. Otherwise, control transfers to a step822. At the step 822, the renderable copy of the content fast mobilemail message (discussed elsewhere hereon) is opened in the regular mailsoftware on the mobile device. After the step 822, processing proceedsto a step 823, where the user goes through an ordinary activity ofviewing and exploring the content of the regular mail message (asimplified version of the full fast mobile mail message). After the step823, processing is complete.

At the step 824, the user opens a full copy of the fast mobile mailmessage in the device software installed on the mobile device. After thestep 824, processing proceeds to a test step 825, where it is determinedwhether an introductory abstract to the message is present, as explainedelsewhere herein (see, for example, FIG. 5, described above). If theabstract is absent then processing proceeds to a step 828. Otherwise,processing proceeds to a test step 826 where it is determined whetherthe user wants to play the abstract for the fast mobile mail message. Ifso, then processing proceeds to a step 827 where the abstract isreplayed to the user. Otherwise, processing proceeds to the step 828.Note that the step 828 also follows the step 827. At the step 828, theuser views, navigates and explores content using regular and specialmeans, such as progressive scrolling buttons 210 on FIG. 2A. After thestep 828, processing proceeds to a test step 829 where it is determinedwhether the user wants to search by fast mobile mail features. If notthen processing proceeds to a test step 832. Otherwise, processingproceeds to a step 830, where the user enters search terms, includingpossibly categories of fast reply tokens, as explained in more detail inconnection with FIGS. 2B-2C and in the accompanying text. After the step830, processing proceeds to a step 831 where the device softwaredisplays all features satisfying the search terms, such as fast replytokens and other fragments of the fast mobile mail message. After thestep 830, processing proceeds to the test step 832 where it isdetermined whether the user has finished content replay, navigation andsearch. If so, then processing is complete. Otherwise, processingproceeds back to the step 828 for further viewing, navigation andexploring of the content.

Referring to FIG. 8D, a flow diagram 840 illustrates in more detailpublishing and delivering complex content created using authoringsoftware with networking capabilities when the content may not be fullyviewable and/or editable in regular mail software, such as fast mobilemail published at the step 807 of FIG. 8A. Processing starts at a step841 where a sender creates a new original content or modifies anexisting piece of content using authoring software, such as fast mobilemail or possibly other authoring software. After the step 841,processing proceeds to a test step 842, where it is determined whetherthe sender (author, user) wants to provide online access to the contentor just wants to save the content locally on the device where thecontent was created. If the sender does not want to provide onlineaccess then processing is complete. Otherwise, processing proceeds to astep 843 where authoring software stores the content online. In case offast mobile mail, such storage may be provided within a cloud service,as explained elsewhere herein. After the step 843, processing proceedsto a step 844, where the authoring software creates a custom URI (link)for the content published at the step 843. As explained elsewhere hereinand illustrated in FIG. 7 and the accompanying text, the custom URI mayinclude a sender's routing ID to allow identifying the sender as theauthor of the published copy of the content and to communicate with thesender concerning the content.

After the step 844, processing proceeds to a step 845, where the senderdesignates recipients of the published content. After the step 845,processing proceeds to an optional step 846, corresponding to anembodiment of the system where the authoring software augments theinitial custom URI provided at the step 844 with routing IDs of therecipients. After the step 846, processing proceeds to a step 847, wherethe authoring software interoperates with mail client or othercommunications software to create a deliverable email message, aninstant message or other medium with custom URI included. Such a messagemay or may not need the content to be rendered in the body of themessage; thus, mail systems may render content in mail client softwarewhen possible but many instant messaging systems may not, for the sakeof brevity of each step of communications. Accordingly, following thestep 847, processing proceeds to a test step 848, where it is determinedwhether the content has to be rendered, at least partially, in the bodyof the message prepared for delivery to the recipients at the step 847.If so, then processing proceeds to the step 849. Otherwise, processingproceeds to the step 850. At the step 849, authoring software creates arendering approximation of the full (published) content, suitable forrendering capabilities of the regular mail, as explained in more detailsin conjunction with the FIG. 7. The software then adds the datarepresenting such approximation to the message. After the step 849,processing proceeds to the step 850, where the message is sent torecipients.

Following the step 850, the remaining steps of the flow diagram 840pertain to a recipient of the message distributed by the authoringsoftware, such as the fast mobile mail. After the step 850, processingproceeds to a step 851, where a recipient opens the received email orother communication (such as an instant message, a tweet or a newlyposted Facebook entry), optionally views the rendered content of themessage available in the body of the message (if such content has beenadded at the step 849) and decides whether to use the custom URI presentin the message. After the step 851, processing proceeds to a step 852,where a recipient's copy of the authoring software parses the customURI, extracts content location and downloads full content. After thestep 852, processing proceeds to a step 853, where the recipient viewsand optionally modifies the downloaded (original) content. After thestep 853, processing proceeds to a test step 854, where it is determinedwhether the recipient is interested in transmitting the modified contentback to sender. If not, then processing is complete. Otherwise,processing proceeds to a step 855 where the recipient's softwareextracts a sender's routing ID from the custom URI and adds the sender'srouting ID to the list of recipients of the modified content. After thestep 855 is a step 856 where the recipient and the original senderexchange roles and the current recipient/author becomes another sender.Following the step 856, processing is complete.

Various embodiments discussed herein may be combined with each other inappropriate combinations in connection with the system described herein.Additionally, in some instances, the order of steps in the flowcharts,flow diagrams and/or described flow processing may be modified, whereappropriate. Subsequently, elements and areas of screen described inscreen layouts may vary from the illustrations presented herein.Further, various aspects of the system described herein may beimplemented using software, hardware, a combination of software andhardware and/or other computer-implemented modules or devices having thedescribed features and performing the described functions. The mobiledevice may be a cell phone, although other devices are also possible.Note that the system described herein may work with a desktop, a laptop,and/or any other computing device in addition to a mobile device.

Software implementations of the system described herein may includeexecutable code that is stored in a computer readable medium andexecuted by one or more processors. The computer readable medium may benon-transitory and include a computer hard drive, ROM, RAM, flashmemory, portable computer storage media such as a CD-ROM, a DVD-ROM, aflash drive, an SD card and/or other drive with, for example, auniversal serial bus (USB) interface, and/or any other appropriatetangible or non-transitory computer readable medium or computer memoryon which executable code may be stored and executed by a processor. Thesystem described herein may be used in connection with any appropriateoperating system.

Other embodiments of the invention will be apparent to those skilled inthe art from a consideration of the specification or practice of theinvention disclosed herein. It is intended that the specification andexamples be considered as exemplary only, with the true scope and spiritof the invention being indicated by the following claims.

What is claimed is:
 1. A method, comprising: at an electronic devicehaving a display, one or more processors and memory, the memory storingone or more programs for execution by the one or more processors:receiving, from an author, an electronic message; presenting, to a userof the device, a selectable set of fast reply tokens that correspond tosnippets of multimedia content to compose an annotated reply electronicmessage; receiving, from the user of the device, an indication of aselection of one or more of the fast reply tokens; providing, to theuser of the device, a mechanism to superimpose a first snippetcorresponding to the selected fast reply token of the one or more fastreply tokens on to different portions of the electronic message, whereinthe fast reply tokens are separate from text within the annotated replyelectronic message; and in accordance with the selection, generating theannotated reply electronic message with the one or more fast replytokens superimposed on the selection; sending, to the author of theelectronic message, the annotated reply electronic message.
 2. Themethod of claim 1, further comprising providing cultural adaptations andlocalization of the fast reply tokens based on a setting provided by theauthor of the electronic message.
 3. The method of claim 1, wherein themechanism to superimpose the selection of one or more fast reply tokensincludes dragging each of the selected one of the fast reply tokens onto a portion of the annotated reply electronic message.
 4. The method ofclaim 1, further comprising superimposing a scroll bar for navigatingthrough the one or more fast reply tokens on to the received electronicmessages.
 5. The method of claim 1, further comprising providing, to theuser, a search bar for searching the selectable set of fast reply tokensby category.
 6. The method of claim 1, further comprising automaticallyrestricting the user from sending predetermined inputs in accordancewith a determination that the user is within a predefined location. 7.The method of claim 6, further comprising automatically restricting theuser from receiving predetermined inputs in accordance with thedetermination that the user is within a predefined location.
 8. Themethod of claim 7, further comprising detecting the user is a driver ina moving vehicle, wherein the user is restricted to sending andreceiving audio-only input in response to the detecting of the user isthe driver in the moving vehicle.
 9. The method of claim 7, furthercomprising: automatically converting the annotated reply electronicmessage from a first type of input to one of the predetermined inputs,wherein the first type of input is distinct from the predeterminedinput.
 10. An electronic device, comprising: one or more processors;memory, the memory storing one or more programs configured to beexecuted by the one or more processors, the one or more programscomprising instructions for: receiving, from an author, an electronicmessage; presenting, to a user of the device, a selectable set of fastreply tokens that correspond to snippets of multimedia content tocompose an annotated reply electronic message; receiving, from the userof the device, an indication of a selection of one or more of the fastreply tokens; providing, to the user of the device, a mechanism tosuperimpose a first snippet corresponding to the selected fast replytoken of the one or more fast reply tokens on to different portions ofthe electronic message, wherein the fast reply tokens are separatefrom-text within the annotated reply electronic message; and inaccordance with the selection, generating the annotated reply electronicmessage with the one or more fast reply tokens superimposed on theselection; sending, to the author of the electronic message, theannotated reply electronic message.
 11. A non-transitorycomputer-readable storage medium storing one or more programs, the oneor more programs to be executed by an electronic device with a display,one or more processors, and memory, the one or more programs comprisinginstructions for: receiving, from an author, an electronic message;presenting, to a user of the device, a selectable set of fast replytokens that correspond to snippets of multimedia content to compose anannotated reply electronic message; receiving, from the user of thedevice, an indication of a selection of one or more of the fast replytokens; providing, to the user of the device, a mechanism to superimposea first snippet corresponding to the selected fast reply token of theone or more fast reply tokens on to different portions of the electronicmessage, wherein the fast reply tokens are separate from-text within theannotated reply electronic message; and in accordance with theselection, generating the annotated reply electronic message with theone or more fast reply tokens superimposed on the selection; sending, tothe author of the received electronic message, the annotated replyelectronic message.
 12. The computer-readable storage medium of claim11, further comprising providing cultural adaptations and localizationof the fast reply tokens based on a setting provided by the author ofthe electronic message.
 13. The computer-readable storage medium ofclaim 11, wherein the mechanism to superimpose the selection of one ormore fast reply tokens includes dragging each of the selected one of thefast reply tokens on to a portion of the annotated reply electronicmessage.
 14. The computer-readable storage medium of claim 11, furthercomprising instructions for determining a tone of the annotated emailmessage based on the fast reply tokens superimposed on to the annotatedelectronic message; and providing to the author, an indication of theoverall tone of the annotated reply electronic message.
 15. Thecomputer-readable storage medium of claim 14, wherein the overall toneof the annotated electronic message is positive and the indication ofthe overall tone is of a smiling emoticon.
 16. The computer-readablestorage medium of claim 11, further comprising instructions forincluding context information with at least some of the fast replytokens superimposed on the annotated reply electronic message, whereinthe context information includes portions of source documents associatedwith at least some of the fast reply tokens.
 17. The computer-readablestorage medium of claim 11, wherein each of the portions of sourcedocuments includes location information indicating a location of thecorresponding portion.
 18. The computer-readable storage medium of claim17, wherein the location information indicates a page number and aposition on the page for the corresponding portion.
 19. Thecomputer-readable storage medium of claim 18, wherein the position onthe page is descriptive.
 20. The computer-readable storage medium ofclaim 11, wherein the electronic message is an email message.